Implementation of KIDMATCH: A Clinical Decision Support Tool for Diagnosing Pediatric Patients with Multisystem Inflammatory Syndrome and Kawasaki Disease

Multisystem inflammatory syndrome in children (MIS-C) is a novel disease identified during the COVID-19 pandemic that may lead to cardiac dysfunction or death in pediatric patients. Early detection of MIS-C remains a challenge given the lack of a diagnostic test and its clinical similarities to Kawasaki disease (KD) and other acute childhood illnesses. We developed and validated the KawasakI Disease vs Multisystem InflAmmaTory syndrome in CHildren (KIDMATCH) clinical decision support tool for screening patients for MIS-C, KD, or other febrile illnesses. Here we describe the implementation and iterative refinement of KIDMATCH with provider feedback as a web calculator in the clinical workflow within Rady Children's Hospital. Our findings demonstrate KIDMATCH and its underlying artificial intelligence model have clinical utility in aiding clinicians at the time of initial evaluation within the hospital setting to distinguish patients who have MIS-C, KD, or other febrile illnesses.

A machine-learning algorithm for diagnosis of multisystem inflammatory syndrome in children and Kawasaki disease in the USA: a retrospective model development and validation study.

BACKGROUND: Multisystem inflammatory syndrome in children (MIS-C) is a novel disease that was identified during the COVID-19 pandemic and is characterised by systemic inflammation following SARS-CoV-2 infection. Early detection of MIS-C is a challenge given its clinical similarities to Kawasaki disease and other acute febrile childhood illnesses. We aimed to develop and validate an artificial intelligence algorithm that can distinguish among MIS-C, Kawasaki disease, and other similar febrile illnesses and aid in the diagnosis of patients in the emergency department and acute care setting. METHODS: In this retrospective model development and validation study, we developed a deep-learning algorithm called KIDMATCH (Kawasaki Disease vs Multisystem Inflammatory Syndrome in Children) using patient age, the five classic clinical Kawasaki disease signs, and 17 laboratory measurements. All features were prospectively collected at the time of initial evaluation from patients diagnosed with Kawasaki disease or other febrile illness between Jan 1, 2009, and Dec 31, 2019, at Rady Children's Hospital in San Diego (CA, USA). For patients with MIS-C, the same data were collected from patients between May 7, 2020, and July 20, 2021, at Rady Children's Hospital, Connecticut Children's Medical Center in Hartford (CT, USA), and Children's Hospital Los Angeles (CA, USA). We trained a two-stage model consisting of feedforward neural networks to distinguish between patients with MIS-C and those without and then those with Kawasaki disease and other febrile illnesses. After internally validating the algorithm using stratified tenfold cross-validation, we incorporated a conformal prediction framework to tag patients with erroneous data or distribution shifts. We finally externally validated KIDMATCH on patients with MIS-C enrolled between April 22, 2020, and July 21, 2021, from Boston Children's Hospital (MA, USA), Children's National Hospital (Washington, DC, USA), and the CHARMS Study Group consortium of 14 US hospitals. FINDINGS: 1517 patients diagnosed at Rady Children's Hospital between Jan 1, 2009, and June 7, 2021, with MIS-C (n=69), Kawasaki disease (n=775), or other febrile illnesses (n=673) were identified for internal validation, with an additional 16 patients with MIS-C included from Connecticut Children's Medical Center and 50 from Children's Hospital Los Angeles between May 7, 2020, and July 20, 2021. KIDMATCH achieved a median area under the receiver operating characteristic curve during internal validation of 98·8% (IQR 98·0-99·3) in the first stage and 96·0% (95·6-97·2) in the second stage. We externally validated KIDMATCH on 175 patients with MIS-C from Boston Children's Hospital (n=50), Children's National Hospital (n=42), and the CHARMS Study Group consortium of 14 US hospitals (n=83). External validation of KIDMATCH on patients with MIS-C correctly classified 76 of 81 patients (94% accuracy, two rejected by conformal prediction) from 14 hospitals in the CHARMS Study Group consortium, 47 of 49 patients (96% accuracy, one rejected by conformal prediction) from Boston Children's Hospital, and 36 of 40 patients (90% accuracy, two rejected by conformal prediction) from Children's National Hospital. INTERPRETATION: KIDMATCH has the potential to aid front-line clinicians to distinguish between MIS-C, Kawasaki disease, and other similar febrile illnesses to allow prompt treatment and prevent severe complications. FUNDING: US Eunice Kennedy Shriver National Institute of Child Health and Human Development, US National Heart, Lung, and Blood Institute, US Patient-Centered Outcomes Research Institute, US National Library of Medicine, the McCance Foundation, and the Gordon and Marilyn Macklin Foundation.

Prediction of Acute Respiratory Failure Requiring Advanced Respiratory Support in Advance of Interventions and Treatment: A Multivariable Prediction Model From Electronic Medical Record Data.

BACKGROUND: Acute respiratory failure occurs frequently in hospitalized patients and often begins outside the ICU, associated with increased length of stay, cost, and mortality. Delays in decompensation recognition are associated with worse outcomes. OBJECTIVES: The objective of this study is to predict acute respiratory failure requiring any advanced respiratory support (including noninvasive ventilation). With the advent of the coronavirus disease pandemic, concern regarding acute respiratory failure has increased. DERIVATION COHORT: All admission encounters from January 2014 to June 2017 from three hospitals in the Emory Healthcare network (82,699). VALIDATION COHORT: External validation cohort: all admission encounters from January 2014 to June 2017 from a fourth hospital in the Emory Healthcare network (40,143). Temporal validation cohort: all admission encounters from February to April 2020 from four hospitals in the Emory Healthcare network coronavirus disease tested (2,564) and coronavirus disease positive (389). PREDICTION MODEL: All admission encounters had vital signs, laboratory, and demographic data extracted. Exclusion criteria included invasive mechanical ventilation started within the operating room or advanced respiratory support within the first 8 hours of admission. Encounters were discretized into hour intervals from 8 hours after admission to discharge or advanced respiratory support initiation and binary labeled for advanced respiratory support. Prediction of Acute Respiratory Failure requiring advanced respiratory support in Advance of Interventions and Treatment, our eXtreme Gradient Boosting-based algorithm, was compared against Modified Early Warning Score. RESULTS: Prediction of Acute Respiratory Failure requiring advanced respiratory support in Advance of Interventions and Treatment had significantly better discrimination than Modified Early Warning Score (area under the receiver operating characteristic curve 0.85 vs 0.57 [test], 0.84 vs 0.61 [external validation]). Prediction of Acute Respiratory Failure requiring advanced respiratory support in Advance of Interventions and Treatment maintained a positive predictive value (0.31-0.21) similar to that of Modified Early Warning Score greater than 4 (0.29-0.25) while identifying 6.62 (validation) to 9.58 (test) times more true positives. Furthermore, Prediction of Acute Respiratory Failure requiring advanced respiratory support in Advance of Interventions and Treatment performed more effectively in temporal validation (area under the receiver operating characteristic curve 0.86 [coronavirus disease tested], 0.93 [coronavirus disease positive]), while achieving identifying 4.25-4.51× more true positives. CONCLUSIONS: Prediction of Acute Respiratory Failure requiring advanced respiratory support in Advance of Interventions and Treatment is more effective than Modified Early Warning Score in predicting respiratory failure requiring advanced respiratory support at external validation and in coronavirus disease 2019 patients. Silent prospective validation necessary before local deployment.

Development and Prospective Validation of a Deep Learning Algorithm for Predicting Need for Mechanical Ventilation.

BACKGROUND: Objective and early identification of hospitalized patients, and particularly those with novel coronavirus disease 2019 (COVID-19), who may require mechanical ventilation (MV) may aid in delivering timely treatment. RESEARCH QUESTION: Can a transparent deep learning (DL) model predict the need for MV in hospitalized patients and those with COVID-19 up to 24 h in advance? STUDY DESIGN AND METHODS: We trained and externally validated a transparent DL algorithm to predict the future need for MV in hospitalized patients, including those with COVID-19, using commonly available data in electronic health records. Additionally, commonly used clinical criteria (heart rate, oxygen saturation, respiratory rate, Fio2, and pH) were used to assess future need for MV. Performance of the algorithm was evaluated using the area under receiver operating characteristic curve (AUC), sensitivity, specificity, and positive predictive value. RESULTS: We obtained data from more than 30,000 ICU patients (including more than 700 patients with COVID-19) from two academic medical centers. The performance of the model with a 24-h prediction horizon at the development and validation sites was comparable (AUC, 0.895 vs 0.882, respectively), providing significant improvement over traditional clinical criteria (P < .001). Prospective validation of the algorithm among patients with COVID-19 yielded AUCs in the range of 0.918 to 0.943. INTERPRETATION: A transparent deep learning algorithm improves on traditional clinical criteria to predict the need for MV in hospitalized patients, including in those with COVID-19. Such an algorithm may help clinicians to optimize timing of tracheal intubation, to allocate resources and staff better, and to improve patient care.